Handling Missing Values when Applying Classication Models

Much work has studied the e¤ect of di¤erent treatments of missing values on model induction, but little work has analyzed treatments for the common case of missing values at prediction time. This paper …rst compares several di¤erent methods— predictive value imputation, the distribution-based imputation used by C4.5, and using reduced models— for applying classi…cation trees to instances with missing values (and also shows evidence that the results generalize to bagged trees and to logistic regression). The results show that for the two most popular treatments, each is preferable under di¤erent conditions. Strikingly the reduced-models approach, seldom mentioned or used, consistently outperforms the other two methods, sometimes by a large margin. The lack of attention to reduced modeling may be due in part to its (perceived) expense in terms of computation or storage. Therefore, we then introduce and evaluate alternative, hybrid approaches that allow users to balance between more accurate but computationally expensive reduced modeling and the other, less accurate but less computationally expensive treatments. The results show that the hybrid methods can scale gracefully to the amount of investment in computation/storage, and that they outperform imputation even for small investments. Keywords: Missing Data, Classi…cation, Classi…cation Trees, Decision Trees, Imputation